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SPECIFICATION 
A METHOD AND SYSTEM FOR CERTIFYING 
OPERATORS OF X-RAY INSPECTION SYSTEMS 

5 [0001] This application claims priority to provisional application Serial No. 



( 60/308,51 0, filed July 27, 2001 , which is hereby incorporated by reference. 

1. FIELD OF THE INVENTION 

[0002] The field of the invention generally relates to x-ray inspection systems 
10 used for security purposes. More particularly, the invention relates to a method and 
jjj system for testing and certifying operators of such systems. 

m 

O 2. BACKGROUND OF THE INVENTION 

m 

* [0003] X-ray inspection systems, e.g., baggage scanners, are commonly used to 
detect and prevent the passage of prohibited items beyond a security checkpoint. Such 

w 

5 X ^ ' ns P ec *' on s y s * ems are iypicaiiy used ai airports, courthouses and other iocaiions 
where security is necessary or desirable. These inspection systems guard against 
items such as weapons and explosives from entering the restricted area. Generally, 
objects are placed on a conveyor and pass through the system at which time they are x- 

20 rayed. An x-ray image of the object appears on a monitor and is viewed by an operator. 

[0004] Regardless of the technology used in x-ray inspection systems, the 

performance of such systems in large part relies on the vigilance and skillfulness of the 
system operators who examine x-ray images of the contents of baggage on the 
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system's video monitor. In turn, the vigilance and skillfulness of the system operators 
generally depend on the amount and type of training they receive. This is very 
important since operator performance is critical to the overall integrity of security 
inspection systems. 

[0005] Traditional methods of training and certifying system operators have 

involved training and testing in a classroom environment, with the use of photographs, 
slide projectors and perhaps computer-generated images. For example, photographs of 
video monitor images showing a weapon or other threat have been used to educate 
system operators on how a threatening object might appear on the system video 
monitor. After receiving such classroom training, system operators have then typically 
been tested in the same classroom setting. However, a classroom environment for 
training and testing is typically very different from the actual setting in which operators 
must detect prohibited items for at least several reasons. 



fy 

□ [0006] First, in the classroom environment, the system operator typically sits in a 

xzs: 
lilsji 

3:5 chair and looks at various photographs and other materials. This does not accurately 
IB- 
replicate real life conditions whereby the system operator must continuously watch the 

system's video monitor, stop and start the system and deal with disturbances. 

Furthermore, real life conditions involving a steady stream of people passing through a 

security checkpoint at an airport, many of whom may be hostile because they are late 

20 for their flights, are not typically replicated in a classroom environment. So despite a 

system operator's passing a classroom test with high marks, that is no guarantee of 

good performance by that operator in the field. 
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[0007] Second, the classroom environment does not effectively teach or test the 

operator's vigilance, i.e., focus and attention span, over time. As noted above, the 
system operator's vigilance is integral to the ongoing success of the security inspection 
process. Indeed, high vigilance during the early portion of an operator's shift during 
5 which time no threatening objects pass through undetected does not excuse a decline 
in attention and focus during the later portion of that operator's shift during which time a 
threatening object passes through. 

[0008] Third, other realities of classroom training further reduce its effectiveness. 
To this end, classroom training and testing may expose system operators to a smaller 
P0 variety of images and threats than the operator would see in real life situations. 

g 

Furthermore, the expense and logistics of classroom training generally result in less 



training and testing than should occur. 



jU : [0009] For these and other reasons, classroom-based training and testing is not 

RJ 

p an optimum or even reliable method of training, testing and/or certifying the operators of 

>$*■■ 

HP x-ray inspection systems. In view of these shortcomings, other training tools have been 

fyi 

developed. 

[0010] One such tool is Threat Imaging Projection (TIP) technology. With TIP 

technology, x-ray images of threat objects are generally merged into the image of non- 
threat baggage that are displayed on the system's monitor. The threat images are 
20 selected at random from a library of various types of prohibited items. An operator's 
performance in detecting threats, missing threats and creating false alarms (detecting 
what is believed to be a threat when there actually is no threat) are then recorded. 
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[0011] Initial TIP technology implementations were limited to simply blending x- 

ray images of threats into the stream of baggage images. If the operator detected the 
threat and activated the appropriate control, the threat object was erased and feedback 
was provided to the operator indicating that he or she had successfully detected the 
5 threat. However, such TIP implementations never adequately worked in the field. 

[0012] For example, TIP technology could not be efficiently used at actual 

checkpoints because the images generated by the TIP technology for training purposes 
would generally be indistinguishable from true threats. Where the TIP technology was 
set up to be automatic, i.e., without input to help distinguish between test and true 

Sb threats, operators often reacted to test threats as though a true threat actually existed. 

0 

^ For example, an operator's detecting a test threat image representing a bomb has 

S3 

mi 

p resulted in that operator calling a bomb squad. These types of events resulted in large 

Mi 

n costs, delays and inconvenience. Where human supervisors were used to overcome 

b 

PJ problems associated with automatic testing, the testing method still proved to be 

□ 

cumbersome and ineffective due to cost logistical problems and potential coaching of 
the operator by the supervisor. 



[0013] More recently, a TIP process was developed that was somewhat more 

successful to a limited degree. This more recent process: 1) employed automatic 
testing, 2) avoided the above-mentioned costs and delays by providing sufficient 
20 feedback to the operator to avoid the situation where a test threat was treated as a true 
threat and 3) recorded individual operator performance in detecting threats so as to 
track performance over extended periods. 
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[0014] However, there still existed no method for using the information provided 
by testing with the use of TIP technology in order to certify operators. Indeed, varying 
uses and interpretation of the data in different segments of the security inspection 
industry results in varying levels of system operator proficiency and hardly any type of 
uniform certification method. Accordingly, there exists a serious need for a structured, 
uniform process for certifying system operators. 



3. SUMMARY OF THE INVENTION 

[0015] The current invention generally involves a method and system for using 
00 TIP technology and the operator testing data it generates in order to certify operators 

5 

® and thereby assure an acceptable and uniform level of operator performance. More 

S3 

ji; particularly, a method and system is described in which the operator views x-ray images 

m 

of objects on a screening system equipped with TIP technology. The system is capable 

N 

fU of recording, storing and transmitting individual operator performance data relating to 

3 . 
detections, missed detections or false alarms. 

s 

m 

[0016] In another aspect of the invention, the method and system provide a 
means to track the types of threats a system operator has seen and detected, as well as 
the difficulty of the threats the operator has seen. To this end, the system contains a 
library of various threat images, e.g., guns, bombs, knives, etc., which are classified 
20 according to type of threat and difficulty of detection. 

[0017] In another aspect of the invention, the method and system provide a 

means for reporting the operator's performance in successfully detecting test images to 
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the appropriate authority, e.g., civil aviation authorities, airlines and other pertinent 
organizations. To this end, the system records the results of the operator's 
performance in a database. These results are then compared to the certification criteria 
that may be contained in another database to determine whether or not the operator 
5 should be certified. Results are reported to appropriate authorities or other 
organizations. 

[0018] In another aspect of the invention, software is provided to accomplish the 
foregoing. More particularly, software is provided to encode images and aid in their 
comparison to real threats. Software is also provided such that the information 
Jo contained in the above-referenced (and other) databases may be accessed and used 



for training and certifying operators. 



□I 4. BRIEF DESCRIPTION OF THE DRAWINGS 

W 

D [0019] Figure 1 is a perspective view of the system of the current invention. 



[0020] Figure 2 is an overview schematic of the system of the current invention. 

[0021] Figure 3 is an on-screen view of a system operations menu; 

[0022] Figure 4 is a flowchart/schematic of a preferred embodiment of the training 
or certification process wherein the flow of information to and from each element of the 
invention is illustrated. 
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[0023] Figure 5 is a flowchart/schematic of an alternative preferred embodiment 
similar to the one in figure 4, except that certain database(s), software and/or other 
system elements are located at a location remote from the inspection system. 

[0024] Figure 6 is a flowchart showing a methodology of the software associated 
5 with the method and system of the current invention. 



5. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0025] The preferred embodiments will now be described with respect to the 

M drawings. To facilitate the description, any numeral identifying an element in one figure 

□ 

jjjj) generally represents the same element when used in any other figure. The 
jj configurations shown in the figures are for illustrative purposes only, and are not 
intended to limit the scope of the current invention. 



b 

»H A. Description of System Elements 

IvsJ 

% [0026] Figure 1 is a perspective view of an x-ray screening system 10 that 
includes a housing 12 containing a screening section 14 through which objects to be 
screened are passed. Screening section 14 may generally comprise a tunnel through 
which objects pass, and may include an active area where objects are X-rayed and 
tunnel-like extensions on either side of the active area. The tunnel may reflect different 

20 cross-sectional shapes. 

[0027] The housing 12 preferably includes a flat base section 16 so that the 
system 10 may readily rest on a flat surface. A video monitor 18 for displaying x-ray 



LA-207011.1 



7 



PATENT 
267/007 



images of objects located inside the screening section 14, and for displaying 
electronically inserted threat item images, may be attached to a top section 20 of the 
housing 12. The system 10 preferably includes a conveyor 22 for transporting objects 
through the screening section 14. 

5 [0028] An information input device or means, shown in this embodiment as a 
keyboard 24, is preferably attached to a front section 26 of the housing 12. The input 
means 24 preferably allows an operator to log onto and off of the system 10, to control 
the conveyor 22, to indicate when the operator believes that a threat item image 
appears on the monitor 18, and/or to perform various other functions. The input means 

M 

jjjj) 24 may alternatively be a touch-screen system, wherein an operator touches images on 

IP 

a video screen to perform various operations, a voice-activated system, a mouse 

m 

p whereby icons are clicked onto, or any other suitable input mechanism to facilitate 

m 

n performing the various functions of the current invention. Alternatively, multiple input 

D 

fU mechanisms may be used in conjunction with one another to allow an operator to 

D 

!t5 perform various Drocedures. 

P ■ 

rut 

[0029] The elements of system 10 may be arranged in several different 
configurations, and a greater or lesser number of elements may be used in the system 
10. Accordingly, figure 1 is not intended to limit the invention to the specific 
configuration shown. 

20 [0030] Figure 2 shows a schematic view of an x-ray screening system 102 

containing a video monitor 104 that displays x-ray images of objects 108 to an operator 
106. Certain elements in figure 2 are generally similar to elements in figure 1 but may 
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be referenced by different numerals. Typically, objects 108 pass through system 102 
on a conveyor 110. As discussed below, the current invention involves various 
databases for providing, storing and transmitting information. Existing types of 
databases and means for providing communication between such databases may be 
5 used with the invention. 

[0031] The configuration shown in figure 2 is for illustrative purposes only. For 

example, figure 2 depicts different databases, software and other items separately. 
However, a number of these items may be combined, e.g., one database may contain 
different types of information and/or software. Accordingly, figure 2 is not intended to 

jfjp limit the invention to the specific configuration shown. 

ft 

yjjj [0032] Screening system 102 is preferably equipped with TIP technology 1 12 so 

D 

gj that images of various prohibited items may be electronically inserted into the normal 

i 

□ ongoing stream of non-threat images, e.g., passenger baggage, of objects 108 viewed 

ro 

D on monitor 104. The non-threat images may be images of actual objects 108, or 
p alternatively, may be electronically generated images. Thus, an operator may be 
trained on the screening system 102 without having to use any real objects. In figure 2, 
TIP technology 112 is simply shown as a box within screening system 102, but it will be 
apparent to one skilled in the art that suitable software and electronics are associated 
with TIP technology 112. 

20 [0033] Screening system 102 preferably also includes a database 114 to store 

general information associated with TIP technology 112. Database 114 may also 
encompass other databases discussed herein, and to this end, figure 2 is for illustrative 
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purposes only. Screening system 102 also preferably includes a data transmission 
means 116 to transmit operator performance data (and other data) for data collection 
and reporting purposes. In figure 2, TIP database 1 14 and transmission means 1 16 are 
shown separately from TIP technology 112, but these items may generally coincide. 

[0034] System 102 also preferably includes TIP library 118 that may contain 

images of various prohibited items such as guns, bombs, knives, etc. TIP library 118 
preferably contains a database defining the threat types (i.e., defined as a gun, bomb, 
etc.) and the difficulty associated with detecting each type of threat or each image 
orientation. For example, certain types of weapons such as Improvised Explosive 
Devices (IDEs) are generally considered more difficult to detect than weapons such as 
Sjjj handguns and will thus have a higher difficulty rating. TIP library 118 may also contain 

m 

pi images of what may be thought of as ordinary items, but in reality may be used as 

m 

» weapons, e.g., box cutters, personal knives, blades for shaving, etc. 



[0035] The difficulty associated with detecting a given threat may also depend on 
j|j> its orientation or angle of view as projected on the monitor 104. For example, a 
handgun is generally considered more difficult to detect when viewed from behind, 
wherein it may appear to be a simple rectangular piece of metal, as opposed to when 
viewed from the side, wherein the well known shape of a handgun is more readily 
apparent. 

20 [0036] It is preferred that TIP library 1 1 8 contains a sufficient number of different 
certification images to prevent memorization by the operator, which would skew any 
testing or certification effort. It is also preferred that the certification images contained in 
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TIP library 118 be kept secret or otherwise confidential in order to prevent operator 
cheating. To this end, the certification images stored in TIP library 118 may be changed 
from time to time to maintain the integrity of the certification process. 

[0037] The images contained in TIP library 118 for certification purposes are 

preferably approved and regulated by the appropriate certifying body such as the FAA. 
In this manner, the current invention may provide a uniform certification process for 
system operators. The images are also preferably graded according to difficulty of 
detection. For example, the FAA currently recognizes various threat classes and each 
class is generally associated with a certain difficulty level. As discussed in more detail 
below, the invention contemplates that an operator would need to be able to sufficiently 
detect images from each level of difficulty in order to become certified. 



[0038] The database in TIP library 118 also preferably defines the type of 

images that the TIP capability 112 will provide for viewing by the operator. To this end, 
images may generally be classified as certification images or training images, 
jfe Certification images are generally those images used during a certification testing 
procedure, a given percentage of which an operator must preferably detect to become 
certified or retain certification. Training images may be similar to certification images in 
appearance, and are generally used to train operators to detect various threats. 
However, training images may differ from certification images in that they portray 
20 images of different threat objects or images of the same threat object but viewed at a 
different angle or orientation. 
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[0039] Certification images and training images may be kept separately in 
different libraries. To this end, TIP library 118 may contain multiple sub-libraries in 
which the respective types of images are separately contained. Alternatively, two 
separate TIP libraries (not shown in figure 2) may be maintained for this purpose. 
Maintaining the training and certification images separately preferably allows a training 
period or certification period to be more readily set up. 

[0040] The parameters of a given training or certification period (referred to 
generally as a testing period) are preferably set by an authorized user, such as a 
manager of a scanning company or an appropriate certification authority. These 
parameters are generally referred to as test scripts, because they "script" a test that will 
be administered to one or more scanner operators. In a preferred scenario, 



Iff! 

P standardized test scripts may be used across the security industry, or segments thereof. 



[0041] In order to set the test scripts for a given testing period, an authorized user 
logs onto the system and enters an ID and password provided by the appropriate 
authorities. After the authorized user logs on, a menu preferably appears, such as the 
on-screen menu shown in figure 3, from which the user may select a desired operation, 
which, in the case of setting test scripts, may be a TIP Configuration' operation 190. 
Other operations may include system maintenance 191, importing/exporting databases 
192, uploading new TIP images 193, downloading TIP reports 194, viewing TIP reports 
20 195, viewing access reports 196, and any other suitable operations described herein. A 
log out 1 option 197 is also preferably included on the menu. Once the TIP 
Configuration' operation is chosen, the authorized user may take the following steps to 
set the test scripts for a given testing period. 
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[0042] First, a period unit of 'DAY 1 or 'MONTH' may be chosen, preferably from a 
drop-down list appearing on the video monitor 18. If a period unit of DAY is chosen, the 
start date of the testing period may be set for the current date or a future date. If a 
period unit of MONTH is chosen, the start date is preferably set for the first day of the 
current month or of a future month. If a start date is set that falls on a date prior to 
these specified acceptable start dates, a warning message may be displayed and the 
user may then modify the start date. 

[0043] Next, a period length may be chosen, preferably from a drop-down list, or 
by entering a number corresponding to the desired period length via the keyboard 24. 
The period length is generally the length of time that the testing period will run. If a 
period unit of MONTH is chosen, and a period length of 'two' is entered, the test will run 
for two months. If a period unit of DAY is chosen, and a period length of 'two' is 
entered, the test will run for two days. 



[0044] In a preferred embodiment, an option to repeat the test may be given. If 
j|5 the authorized user, or scanner operator (if the operator is given such authority), 
chooses to repeat the test, then the test may be administered over the previous period 
length, or until a new test is scheduled, which preferably overrides the repeat test. For 
example, if the testing period is two months, and the user chooses to repeat the test, 
then the test will be repeated over the next two months, or until the date that a new test 
20 is scheduled to start within that two month period, at which time the new test preferably 
overrides the repeated test. In a preferred embodiment, two testing periods may not 
overlap one another in a given system 10. 
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[0045] A pass percentage, which indicates the test score that an operator must 
achieve to obtain certification as a scanner operator, is also preferably set, either via a 
drop-down list or by entering a percentage number via the keyboard 24. The pass 
percentage may range from 1 to 100. Additionally, a description including any 
5 information relevant to a particular test may optionally be entered by the authorized user 
via the keyboard 24. 

[0046] The library containing certification images may also contain information 
pertaining to the detail of, or difficulty level in identifying, each certification image. The 
library containing training images may generally contain more images to allow operator 
||) training with a variety of images on which to gain experience. The library containing 
m training images may not necessarily specify difficulty levels associated with each 



in 



D 



training image. 



[0047] A certification image list for a given test, or set of tests, may be created or 

m 

D "scripted" by selecting threat item images from a general source threat list and moving 
or dragging the selected items to the certification image list via a mouse connected to 
the keyboard 24, or via any other suitable method. In a similar manner, threat items 
may be removed from the certification image list and placed back into the general 
source threat list. Once the test scripts for a given test, or set of tests, are established, 
the user may save the test scripts in the system memory by clicking on a 'Save Test' 
20 icon displayed on the video monitor 18, or by any other suitable data saving method. 

[0048] Training images may be used during a certification test, i.e., they may be 
randomly merged into the stream of baggage images,. to keep operators alert and 
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prevent them from memorizing which images are certification images. This generally 
allows system 102 to provide a mix of images having varying difficulty levels. By 
providing both training and certification images to the operator during certification, the 
operator preferably does not rely solely on his or her memory of what the certification 
images may look like. However, training images are preferably not used as images that 
will actually count towards certification since operators may come to memorize training 
images during the training process. 

[0049] Screening system 102 also preferably includes an operator performance 
database (OPD) 120. OPD 120 preferably contains comprehensive records of test 
results and performance data for each individual operator 120 that operates system 
102. Information that may be stored in OPD 120 may include the date and time of each 
test, the type of image (along with its difficulty) presented to the operator, and the 
operator's test results, including whether the operator achieved a score at or above the 
specified pass percentage. To this end, information regarding the operator's 106 
detections, non-detections, and false alarms, as well as the amount of time an operator 
spent evaluating a particular image(s), may be stored in OPD 120. Grades for the 
operator's 106 performances may also be stored in OPD 120. It should be noted that 
other information pertaining to operator performance may also be stored in OPD 120. 

[0050] There may be different levels or classes of certification requiring various 
different levels of proficiency and/or experience. Accordingly, the operator's 106 levels 
of proficiency and experience over time may also be stored in OPD 120. OPD 120 thus 
preferably provides information on the operator's progress over time. This allows 
evaluation of how an operator or group of operators perform or progress over time. 
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[0051] Where applicable, system 102 may also serve to promote an operator 

106 to a higher level of certification by virtue of the information in OPD 120 reflecting 
that operator's performance. In this manner, system operator 106 may be allowed to 
operate system 102 at times or during conditions requiring this higher level of 
certification. To this end, system 102 may compare the operator's log in information to 
the information stored in OPD 120 to determine whether the operator is indeed certified 
to operate system 102 at that time or during the existing conditions. Accordingly, the 
current invention preferably provides a safety check to ensure that properly certified 
operators are operating the system 102 at appropriate times. 



[0052] Screening system 102 also preferably includes a certification criteria 

database (CCD) 122 which may generally contain and set forth requirements for 
operator certification. To this end, CCD 122 may set forth different sets of requirements 
that are required to obtain different types of certification. As such, CCD 122 may 
specify the number of images, types of images and levels of difficulty of images that 
lH must be successfully detected by an operator 106 during a certification test. CCD 122 
$ may specify the length of time over which a certification test is to be conducted or the 
maximum time allowed for an operator 106 to complete a certification test. CCD 122 
may set forth the frequency of testing necessary to maintain certification. CCD 122 may 
also specify the criteria for determining whether an operator's performance constitutes 
20 an acceptable/certifiable performance (success criteria). 

[0053] It is preferred that CCD 122 set forth requirements that are uniform for a 

given industry or application. For example, with certification for airport baggage 
scanner operators, it is preferred that criteria contained in CCD 122 be uniformly used 



LA-207011.1 



16 



III! 
SI 

US 



rif 



PATENT 
267/007 

to ensure a minimum certification level throughout the industry. Such uniform criteria 
may be specified by the appropriate authority such as the FAA or other civil aviation 
authority. 

[0054] The criteria contained in CCD 122 may also be changed over time to 
reflect changes in desired certification requirements. For example, should a new type of 
weapon come into being, the certification criteria stored in CCD 122 may be revised to 
include that image as part of the images to be seen by operators during the certification 
process. In this situation, TIP library 118 may also be updated to include this new type 
of image as a training and/or certification image. 

[0055] System 102 also preferably includes software 130 that accesses 

information contained in the various databases described above and that analyzes 
operator performance during testing and certification. In figures 1-2, software 130 is 



D referred to as Threat Image Projection - Certification Version software (TIP CV), though 

m . 

D this name is not intended to limit the scope of the invention. More particularly, 



certification software 130 may access information in the various databases, receive and 
record information about operator performance, analyze operator performance and 
provide reports thereon. 

[0056] To accomplish the foregoing objectives, certification software 130 may 
perform various functions. For example, software 130 may control which images are 
20 presented to the operator 106 from TIP library 112. This may be accomplished via the 
test scripts inputted by a user, as described above, or through a random selection of 
images. It is preferred that a sufficient number of images from varying levels of 
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detection difficulty are presented to the operator so as to render the training and/or 
certification meaningful. To this end, software 130 may retrieve certification images 
from TIP library 112 according to the criteria set forth in CCD 122. And as mentioned 
above, these criteria may change over time. 

5 [0057] Certification software 130 may also control the timing of the images 
presented to the operator 106, via the test scripts inputted by a user or via a random 
timing method. In other words, software 130 may control the timing of when various 
training or certification test images are inserted into the normal stream of images viewed 
by the operator 106. For example, software 130 may insert several training or 

j|b certification test images in a row, near each other or at a time apart from each other. 

Sjjj This preferably enhances the training and certification capability of system 102 by 

iJi 

p simulating the randomness by which threats may pass through system 102 in real life, 
pi [0058] Software 130 may also ensure that the appropriate number of certification 

ii 

D images are shown to operator 106 within the time period specified in CCD 122 for that 

55 type of certification. This may occur by increasing the frequency of test images 

RJ 

displayed to the operator as the end of the test period nears, if necessary. 

[0059] Software 130 may also record whether an operator 106 successfully 
detects or does not detect the training or test images as well as how much time an 
operator 106 spends evaluating these images. Based on such information, software 
20 130 may determine whether a given operator's performance is acceptable. For 
example, for certification purposes, software 130 may determine whether the operator's 
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performance was sufficient for that operator 106 to become certified or maintain his or 
her certification level. 

[0060] Software 130 may also provide a reporting function on the operator's 
performance to the operator and/or the appropriate authorities, as further described 
below. For training purposes, software 130 may provide information on which training 
images were successfully detected or not detected, and may also provide 
recommendations for further training so as to increase the operator's performance in the 
future. This type of reporting function may help provide for uniform training standards in 
a given industry. 



j|0 [0061] For testing and certification purposes, software 130 may provide a report 
indicating the operator's test score and whether the score suffices for certification. This 
may occur by a comparison of the operator's performance information that may be 
contained in OPD 120 to the certification criteria contained in CCD 122. Software 130 
may also provide a description of what types of images were successfully detected or 
ER> not, and thus provide information on the areas needing further work by the operator. 
This reporting function may also help serve to establish uniform testing guidelines in an 
industry. 

[0062] In determining whether an operator should be certified, software 130 may 
supplement the image and associated difficulty level. Instead of just considering the 
20 image and level of difficulty in and of themselves, software 130 may analyze the entire 
image displayed to the operator containing the threat image as well as its surroundings. 
This may occur because a certification image that is normally easy to detect may be 
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made difficult to detect when placed in a certain location within the ongoing stream of 
non-threat baggage. This may also occur when an image is not projected well on video 
monitor 104 due to insufficient x-ray penetration of the baggage, the image into which 
the certification image was merged. 

[0063] As a result, an operator may fail to detect an otherwise easily detectable 
image and may be graded more harshly than he or she should have been. By 
supplementing the level of difficulty assigned to a given certification image with an 
analysis of the context in which that image is place, the software 130 may more 
accurately evaluate an operator's abilities. 
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B. Description of Training and Certification Processes 
[0064] The process of training and certifying system operators is now discussed 
in more detail with continued reference to figure 2, but also with reference to figures 3 
and 4. Figures 3 and 4 are schematics generally depicting the training and certification 
method of the invention. The configuration shown by these figures is not intended to 
limit the scope of the invention. 

[0065] As indicated above, the system and method of the current invention may 
provide for uniform training and certification throughout an industry. The current 
invention also provides for regulation of the certification process to avoid cheating. And 
20 because the information contained in the various databases may be changed to reflect 
changing conditions and requirements facing operators, the current invention provides 
for flexibility in the certification process. 
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[0066] In one embodiment of the invention, an operator may be randomly tested 

for certification purposes while generally performing his or her screening duties. That is, 
any time the operator is screening baggage or other objects, a test or certification 
process may occur. This may occur with or without the operator's knowledge. Where 
the testing or certification occurs without the operator's prior knowledge, the situation is 
avoided whereby an operator maintains focus and attention only when a known testing 
period is occurring. 

[0067] However, there may be situations where the operator is advised in 

advance that the screening system 102 will be providing images to the operator and/or 
recording operator performance. For example, system 102 may provide training images 
so that the operator may be trained on how to detect or not detect certain threats. In 
this situation, it may be preferred to advise the operator in advance that the system will 

i 

i be providing training images in order to allow effective and constructive training. Once 

Ci 

m sufficient training has occurred, however, the operator may be randomly tested without 

0 

iff* nrinr knowleHne that a test is to nr.r.nr. 

rii 

[0068] An operator may log into screening system 102 as shown in step 160a at 
the beginning of his or her screening shift, training period, testing period or other period 
during which the operator is operating system 102. Suitable identification and password 
information may be issued to each operator for the logging in process. In this manner, 
20 system 102 may record the identity of the operator who is performing the screening 
duties at a particular time. The identification and password method of logging in also 
helps to avoid cheating in the training and certification process. For example, it 
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preferably prevents a more skilled operator from filling in for a less skilled operator. In 
similar fashion, operator 106 may log off from system 102 as shown in step 160b. 

[0069] Upon an operator logging on to system 102, software 130 may 

acknowledge the identity of the operator who will be viewing the images. In this 
manner, information in OPD 120 may determine whether the operator logging at the 
time is certified to operate system 102. It should be noted that it is not absolutely 
necessary for software 130 to acknowledge the operator's identity for training or testing 
to occur. Also, it is not absolutely necessary for the operator to log on for software 130 
to provide images. Accordingly, it is not intended that the invention be limited as such. 

[0070] Some or all of the screening systems 102 in a given facility or facilities, 
may be networked so that a given operator 106 can log on to any screening system 102 
on that network. Some or all of the screening systems 102 may have access to the 
relevant databases so that a training session, certification test or portion of a 
certification test could be conducted at different screen system 102 locations. This 
provides flexibility in that a given operator could be assigned to different screening 
systems 102 during a certification test without affecting the certification process. 
Similarly, the problem of having only one screening system 102 in a facility that is 
capable of certifying a given operator 106 is avoided, as is the situation where operators 
maintain focus and attention only when stationed at that particular system 102. 

20 [0071] Software 130 may provide training and/or testing images depending on 
the circumstances. For example, software 130 may access CCD 122 to determine 
which images should be shown according to the specified criteria, and then access TIP 
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library 1 18 to access the desired images to provide to operator 106. As shown in figure 
4, the portion of software 130 that interfaces with TIP library 1 18 is generally referred to 
as image control software 1 30a. 

[0072] When system 102 is being used for training purposes, image control 
software 130a may access a broad range of images which allow the operator to become 
familiar with all types of images that might be encountered later on during testing and 
certification, as well as in real life. The length of a training session may be controlled by 
software 130, preferably via the 'period unit' and 'period length' test scripts that are 
entered by an authorized user, as described above. In this manner, system 102 may be 
able to correlate the amount of training with the operator's later success when being 
tested. 

[0073] In the testing environment, it may be preferred that certification testing of 
an operator 106 occurs over a set interval of time whereby test images are provided for 
viewing on monitor 106. The length of the test interval may be specified in the 
certification criteria database 122, or via the 'period unit' and 'period length' test scripts 
entered by an authorized user. In this manner, testing for certification may achieve 
better uniformity because operators that are not efficiently able to correctly identify test 
images will not have some indefinite period of time to do so which itself would indicate 
that the operator is not worthy of certification. 

20 [0074] When system 102 is being used for certification purposes, the images 
accessed by image control software 130a and provided to the operator 106 may be 
determined by the criteria contained in CCD 122. The images to be provided to 
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operator 106 may be set forth in a pre-selected list or script file which may be 
randomized by software 130a. This preferably helps to prevent memorization and/or 
sharing of the specific test sequence of images. 

[0075] Software 130 may also generate automatic test sequences using criteria 

5 specified in CCD 122 such as the types of images, number of images, frequency of 
projection and length of test. That is, criteria may be used that call for certain types of 
images, a certain number of images and the frequency at which they are shown. 
Software 130 may then use these criteria to automatically generate a test sequence 
having a string of certification images according to these criteria. 

[si;: 

D 

So [0076] Automatic generation of test sequences preferably reduces the burden of 



M 



having to pre-program a certification test sequence of images. Indeed, by creating 
multiple groups of test sequences with varying difficulty levels, system 102 simplifies the 
task of generating image lists for appropriate certification levels. 



[0077] Once an operator 106 has completed a test sequence, the software 130a 

□ 

'l% may automatically select a different test sequence for the next test. These subsequent 
sequences may be controlled by an appropriate certification control office or by allowing 
software 130a to automatically generate a random subsequent test sequence that has 
not been previously seen by the particular operator 106. 

[0078] As mentioned above, it is preferred that certain images of certain 

20 difficulties are provided to the operator 106 when testing for certification purposes. For 
example, it is preferred that during a testing/certification interval, at least one image of 
each type of image requiring certification appears on the monitor. To this end, image 
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control software 130a may first access CCD 122 to retrieve the pertinent certification 
regulations and protocols, and for instructions on which images to access from TIP 
library 118. 

[0079] Thereafter, image control software 130a may provide the testing images 

5 to the operator based on these instructions. In addition to the type of images provided, 
image control software 130a may also receive instructions on the frequency that each 
test image will be provided to the operator 106. In this manner, the test images viewed 
by the operator 106 may be varied to simulate real life conditions. 

[0080] As the operator 106 views the monitor 104, images from TIP library 118 

P 

Gib appear. In the training situation, it may be that various training images appear one after 
the other so as to allow the operator 106 to become familiar therewith. In the testing 

y 

j^! and certification situation, test images may be inserted into the ongoing stream of non- 
13, threat baggage or other objects. 

m 
pi 

I; [0081] As the operator 106 views the stream of images on monitor 104 ; he or 

D 

IH5 she will generally detect (170a), not detect (170b) or falsely detect (170c) what are (or 
are not) believed to be threat images. The operator's 106 responses, i.e., detect 170a, 
no detect 170b or false detect 170c, are preferably recorded in the operator 
performance database 120 via software 130. Software 130 may use this performance 
information in various ways. 

20 [0082] For example, the information recorded in OPD 120 may be evaluated by 
a portion of software 130 that is designed to do so. In figure 4, this portion of software 
130 is referred to as certification software 130b. Overall, certification software 130b 
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may be used to compare the information recorded in OPD 120 regarding the operator's 
responses to information contained in the CCD 122 that may set forth the correct 
responses. 

[0083] CCD 122 may also set forth the minimum allowable standards for an 

operator to become certified or maintain his or her certification. Such minimums may be 
set by the appropriate authority. Images that are successfully detected may go towards 
meeting the minimum allowable standards. Upon making this comparison, certification 
software 130b may generally determine whether the operator passes 180a and thus 
becomes certified or maintains certification, or whether the operator fails 180b and thus 
does not become certified or loses certification. To this end, software 130b may take 
into account the circumstances surrounding the certification images. For example, if 
unusual difficulty surrounded the image such that it was more difficult to detect than 
would otherwise be, this may be taken into account so that the operator's performance 
is more accurately considered. 



[0084] As another example, the information recorded by OPD 120 may be used 

to determine what further training the operator 106 should undergo. For example, 
software 130 preferably notes all the incorrect responses recorded in OPD 120. If it is 
seen that operator 106 has trouble detecting a certain type of threat image, after the 
training or testing interval has occurred, this information may be used to convey to an 
20 operator (or the appropriate authority) that further training is recommended on that type 
of image. 
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[0085] As another example, the information recorded by OPD 120 may be used 
during a certification interval that is then occurring. To this end, the certification criteria 
contained in CCD 122 may specify that a certain number of different types of images 
must be detected during the certification interval for that operator to pass. If, during the 
5 certification interval, the operator 106 fails to detect a certain type of image, this 
information may be recorded in OPD 120 and then used by software 130 to instruct the 
image control software 130a to provide another image of that type to the operator 106. 

[0086] After a certification interval ends, certification software 130b generally 
advises whether the operator 106 passes 180a or fails 180b. If the operator 106 
fD passes, he or she is preferably notified and thus need not undergo further testing until 



?! some time in the future. Indeed, the operator 106 may not be aware he or she was 
being tested until being made aware that he or she passed. As mentioned above, 
subsequent tests are preferably scheduled in a quasi-random manner so as to maintain 
operator vigilance. 



[0087] The scheduling of subsequent tests may occur per the criteria in CCD 

122. To this end, certain operators may need to be tested more frequently than others. 
Accordingly, it is preferred that system 102 keep track of the type of operator that is 
operating the system so that he or she is tested frequently enough. 

[0088] If operator 106 fails, operator 106 is again preferably notified of the failing 
20 result. This may result in various events. For example, operator 106 may be placed on 
some type of probationary status. Alternatively, operator 106 may fail to become 
certified (or may become decertified) and thus be precluded from operating system 102 
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for the time being. It may be that several failing grades in subsequent attempts to 
become certified, or consistently low scores when attempting to maintain certification, 
results in the operator being given other job assignments. 

[0089] As another example, the results recorded in OPD 120 may be used to 
determine whether a given operator 106 is permitted to log on and operate the 
screening system 102. If an operator has not achieved or maintained certification, the 
operator may be prevented from logging on to and operating the screening system 
absent the intervention of a supervisor. Similarly, if, during the course of a certification 
test, an operator 106 fails to detect a certain number of certification images such that 
the operator 102 cannot pass even with a perfect performance from that point on, the 
operator 106 may be automatically logged out and prevented from further operating the 
screening system 1 02 absent intervention by a supervisor. 



[0090] As another example, the results recorded in OPD 120 may be used to 
create an individualized training regimen for a particular operator 106. By determining 
P> the strengths and weaknesses of each operator 106 based on the type and difficulty of 
each image that was not detected, software 130 may then be used to formulate a 
training regimen that emphasizes those types of images with which operator 106 had 
difficulty. In other words, system 102 may provide a training program that is tailored to 
address a particular operator's weaknesses. This in turn may lead to a passing score 
20 on the next certification attempt, as well as a better trained operator work force. 

[0091] As another example, the results for various operators 106 that are 
recorded in OPD 120 may be evaluated to determine trends in detection. To this end, if 
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a significant number of operators 106 have difficulty detecting a certain type of threat 
image, this information may be used to institute industry-wide training on this type of 
threat. Other types of trends may also be determined by evaluating the information 
contained inOPD 120. 

5 [0092] To achieve this goal, software 130 preferably provides a reporting function 
by accessing and manipulating data contained in TIP library 118, OPD 120 and/or CCD 
122 to provide desired information. To this end, software 130 preferably includes 
various types of filters so that authorized users may extract certain types of information 
from the various databases in a format that provides useful reports. Such reports may 

i%) include threat detection percentages by category, individual screener performance 

D 

ft summaries, performance comparisons of multiple screeners, and any other suitable 

pj information for evaluating system and operator performance. 

§ 

p [0093] The 'threat detection by category' reports preferably include the following 

m . 

D fields: 



• the name and ID number of each selected screener; 

• the dates on which each screener underwent testing; 

• the time within a test at which each image appeared; , 

• the threat category (e.g., gun, bomb, etc., which will be the same for each 
item in a 'threat detection by category' report); 

20 • the subcategory of each threat, i.e., conventional or otherwise; 
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• the threat description (e.g., the specific gun model or bomb type); 

• the decision outcome, i.e., whether the threat was identified ("hit" or "miss"); 

• the recognition time for each identified item, i.e., the amount of time that 
elapsed before the screener identified the threat; 

• the total recognition time for all the threats from a testing period; and 

• the average recognition time for all the threats from a testing period. 

[0094] The 'individual screener performance summary' reports preferably include 
the following fields: 

• the name and ID number of a selected screener; 

• the airport and terminal where the screener underwent testing; 

• the date each report was created; 

• the number of shifts worked by the screener; 

• the number of bags screened by the screener; 

• the number of certification images presented; 

• the number of certification images properly identified; 

• the number of certification images missed; 

• the percentage score of properly identified items; 

LA-207011.1 30 
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• the total and average daily performance of the screener; and 

• whether the screener passed or failed a given test. 
[0095] The 'screener comparison' reports preferably include the following fields: 

• the name and ID number of each selected screener; 

• the number of shifts that each screener worked; 

• the number of bags screened by each screener; 

• the number of certification images presented to each screener; 

• the number of certification images properly identified by each screener; 

• the number of certification images missed by each screener; 

• the percentage score of properly identified items for each screener; 

• whether each screener passed or faiied a given test; 

• the grand total of properly identified items for all selected screeners; and 

• the average performance of all selected screeners as a whole. 

[0096] The TIP reports may be downloaded by clicking on the 'Download TIP 
15 Reports', or similar option, from the main menu. The 'threat detection by category,' 
'individual screener performance summary,' 'screener comparison' and any other 
suitable reports may be downloaded individually, or all of the reports may be 
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downloaded together. To download the reports, a test period and a destination are 
preferably selected by an authorized user. For example, an authorized user may select 
all CV reports administered over a two month period from February 1 to March 31 in a 
given year, and choose to have those reports downloaded to a floppy drive, a zip drive, 
5 a hard drive, or any other suitable medium. If a test period and/or a destination is not 
selected, a warning message will be displayed, and the authorized user may then add 
the required information. The downloaded CV reports are preferably saved to the 
selected destination as text (.txt) files. 

[0097] Certain aspects of the functionality provided by system 102 that have 

fSSt t 

H) been mentioned above bear reiteration in this description of the testing and certification 

Oil 

J method. First, certification images are preferably approved and/or graded for difficulty 

"Hi 

\M 

P by the appropriate authority. Second, certification images are preferably kept secret to 

i 

» avoid cheating by operators seeking certification. Third, it is preferred that certification 

Mi 
bat 1 

W images are sufficiently numerous to avoid memorization by the operators 106. To this 

5! 

+5 end certification imaaes mav be undated over time, but when this occurs, it is again 
! * desired that the appropriate authority approve and/or grade the new images. 

[0098] Referring now to figure 5, an alternative embodiment of the current 
invention is shown wherein like reference numerals are used except that numerals in 
the 200s are used instead of the 100s as in figure 4. 

20 [0099] In this embodiment, the certification software 230b may be physically 
located at a remote location away from screening system 202. Other elements 
described above may also be located remotely to system 102. In this embodiment, 
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software 230b and other components of system 202 may be under the control of, and 
located at, the appropriate authority such as the FAA. As discussed below, this may 
allow the appropriate authority to have more control over evaluating operator responses 
270a-c and determining whether operators become certified (or maintain their 
5 certification). This alternative embodiment is substantially similar to the embodiment of 
figure 4 but may differ as follows. 

[0100] As shown by the dotted line, the operator responses 270a-c recorded in 
OPD 220 may be transmitted to certification software 230b located at the remote 
location via a global secure internet connection, a hierarchical network structure, a 

*§D physical diskette, wireless transmission, or some other means of data transmission. 

D 

The certification criteria database 222 may be present at both the location of the 



pi screening system 202 as well as the remote location. In this manner, CCD 222a 

s 

„ located at the screening system 202 may be used to test an operator 206, while CCD 

Pi 

t!ii 222b located at the remote location may be accessed when determining if an operator 
Dassed. 



Si 
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[0101] More particularly, image control software 230a may access CCD 222a as 
a factor in determining the timing and sequence of images to be displayed on monitor 
208 during the certification process at the screening location. At a separate location, 
certification software 230b may compare the standards contained in CCD 222b with the 
20 responses recorded for each individual operator 206 as transmitted from OPD 220 in 
order to determine whether the individual operator 206 should be certified 280a or not 
certified 280b. This alternative configuration preferably allows the governing body more 
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direct control of the certification process. As shown by the dotted line, updating one of 
CCD 222a or 222b may result in the updating of the other of CCD 222b or 222a. 

[0102] Referring now to figure 6, the functionality of software 130 and how it is 
used with system 102 is now further described. As shown, x-ray system software 312 
may reside on a computer associated with system 102. Software 312 may generally be 
associated with the system's TIP capability (112 in figure 2) and is thus represented by 
the reference numeral 312 in figure 6. Certification software 330 (similar to software 
130 and 230 in figures 1-3) may reside on a server that has connectivity to system 102. 
Indeed, in a preferred embodiment, the current invention is used in a network 
environment. 



2 [0103] Operator 106 may log onto system 102 as shown in step 302. As shown 

W 
n 

pp!j in step 304, certification software 330 preferably generates a certification test script 
pi containing various parameters such as the length of the testing period, the minimum 

s 

□ score required to pass and the images to be used. During the testing period, some 

9B number of certification images may be shown to the operator 106 who must 
Un- 
successfully detect some minimum number to pass. As discussed above, these 

parameters may vary according to criteria contained in CCD 322. Accordingly, there 

may be connectivity with OPD 320 in figure 6 when the test script is being generated as 

shown in step 304. 

20 [0104] The test script may also vary according to the type of certification sought 
by operator 106. As also discussed above, the test script is preferably generated by 
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software 330 automatically. Alternatively, however, test scripts may be programmed by 
the appropriate authority. 

[0105] As shown in step 306, the certification test script 304 may be transmitted 
to system 102. Various types of transmission may be used, but in one embodiment, 
5 transmission may occur electronically over a local area network (LAN). It should be 
noted that the foregoing discussion is not intended to require that steps 302, 304 and 
306 occur in a certain required order every time. 

[0106] A certification test may then generally occur. The certification interval will 

j*& begin and over time, the operator 106 may be shown both certification images and 

D 

O0 standard, or training, TIP images. As discussed above, both types of images may be 

If! 

H j provided to operator 106 to prevent memorization or sharing of the certification images. 
Jjj At some point after the start of the interval, the operator 106 will have seen some 
portion of overall number of certification images set forth from the test script 304. 



£ [0107] A list of remaining certification images to be shown to operator 106 

Pi 

fU)5 (remaining images list or RIL) is preferably generated as shown in step 308. RIL 308 
may be generated at various points in the testing interval. In a preferred embodiment, 
RIL 308 may be generated each time operator 106 logs onto system 102. 

[0108] As shown in step 310, queries may be generated as to how many 
certification images of the original number remain to be seen by operator 106, and how 
20 many days remain in the certification interval. Other queries may be generated. Based 
on these two (or other) factors, it is determined whether a certification image or a 
standard TIP image will be next shown to operator 106. To this end, for example, if 
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there are few days remaining in the certification interval and there are a significant 
number of certification images that remain to be seen by operator 106, it may be that a 
certification image is more likely provided. 

[0109] If the outcome of step 310 is that a certification image is provided as in 
step 312a, a certification image is preferably picked from RIL 308 at random as shown 
in step 314. In connection therewith, that certification image is preferably removed from 
RIL 308. If the outcome of step 310 is that a standard TIP image is provided as shown 
in step 312b, a certification image will not be removed from RIL 308 and step 314 is 
generally bypassed. Upon either a certification TIP image or standard TIP image being 
chosen, the TIP image is merged into the image of the image of the baggage as shown 
in step 316. 



g; [0110] Thereafter, the operator's performance may be recorded as shown in step 
Qj 318. The operator performance may generally comprise a detect, no detect or false 
□ alarm as discussed in connection with steps 170a-c in figure 4. The information 

B 

935 reflecting the operator's performance may be transmitted to operator performance 

m 

database (OPD) 320 as shown in step 330. As discussed above, it is generally 
preferred that the operator's performance on only the certification images count towards 
determining whether certification is obtained or maintained. 

[0111] As the certification interval progresses, operator 106 will be shown more 
20 images in the manner described above. This repetitive process is shown by step 332. 
Generally, it is preferred that operator 106 has the opportunity to see all certification 
images during the certification interval. Alternatively, it is preferred that operator see at 
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least a minimum number of certification images that would enable the operator to 
become certified. 



with the information stored in CCD 322 to determine whether the particular operator 
5 passes. This comparison may occur multiple times until the certification interval is 
complete. To this end, query 334 may be generated on whether the testing interval is 
completed. If not, as shown in step 336a, additional comparison(s) between operator 
performance data in OPD 320 and certification criteria in CCD 322 will occur. 

M [0113] When the certification interval is complete as shown in step 336b, the 

P 

$b operator's overall performance will be assessed according to the minimum score 



^ requirement set forth in the test script 304. If the minimum score is achieved, operator 

in 

Pi 

g 106 becomes certified or maintains certification as shown in step 380a. Alternatively, if 

q operator 106 does not meet the minimum score, certification is not obtained or may be 

JU 

CP lost as shown in step 380b. 

b 

PIS [0114] In an alternative embodiment, the screening system 102 is incorporated 
into a checkpoint wherein various types of other information generated by other 
screening methods or devices are recorded to supplement the information described 
above. In locations such as airports, x-ray screening systems are generally used in 
combination with metal detectors, where the x-ray screening system is used to detect 



20 threats contained in baggage and the metal detector is used to detect threats concealed 
by people under clothing, etc. 



[0112] 



As the operator's performance data is stored in OPD 320, it is compared 
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[0115] The alarms on metal detectors may be activated by a significant 
percentage of people passing through checkpoints, requiring further inspection. Other 
methods and devices may also be used to detect threats and the information generated 
by their use may also be used to supplement the information described above. 

5 [0116] By recording the number of people and bags that pass through a 
checkpoint as well as the number of detections by the metal detector, a more complete 
measure of the total activity at that checkpoint may be ascertained. Accordingly, the 
current invention preferably includes databases and software to record, store and report 
on the volume of people passing through a checkpoint as well as the number of 

5Pb detections made by the metal detector(s) (or other detection devices) at that checkpoint. 

D • 



It is also preferred that the metal detector and/or other detector information may be 
transmitted over a network in similar fashion to the certification information described 
above. This other detection information may be incorporated with the certification 
information to provide an overall picture of operator performance. 



i35 [0117] This activity may be compared with operator performance data from the 
OPD 120 to determine the effect of overall checkpoint activity levels on operator 
performance. For example, operator performance may suffer during periods of 
increased activity at a checkpoint due to distraction. As such, this information may be 
used to determine a minimum number of personnel at a given checkpoint for given 

20 activity levels in order to maintain a specified level of threat detection by operators 106. 

[0118] Information relating to the amount of activity at a checkpoint may also be 
used to determine the timing of certification testing and the timing of the display of threat 
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images during certification testing. For example, certification testing may be avoided 
during periods of high activity because of delays it may cause. Similarly, the number of 
certification images displayed during times of increased activity may be decreased in 
order to lessen delay and increase the flow of people through a checkpoint. 
5 Alternatively, the number of certification images displayed during periods of increased 
activity may be increased in order to test and certify an operator's performance under 
more difficult conditions. 

[0119] While various preferred embodiments and configuration have been 
described above, the current invention is not limited to these exact embodiments and 



tt configuration. For example, various databases or software components separately 

5 

described above may be combined. Accordingly, one skilled in the art will appreciate 

H\ 
ml 

gj that variations may be made to the foregoing description without departing from the 

P9 

g scope of the current invention. 
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